Recorded time-varying content is an ever-increasingly popular form of content. Various types of recorded time-varying content are known, including as examples video (e.g., movies, films, etc., which may comprise audiovisual content), audio (e.g., music, songs, audio-books, etc.), animation (e.g., cartoons, video games, etc.), multimedia content, and other forms of recorded content whose output vary over time. During the playback of such recorded time-varying content, the output changes (e.g., the images being displayed in a video change during the playing of the video, the audio output during playing of a song changes over the course of the song, etc). Thus, recorded time-varying content, as used herein, generally refers to any type of content (e.g., video, audio, etc.) whose output varies over the time of the recording.
Such recorded time-varying content is often stored in electronic form (e.g., as an electronic file) to a computer-readable data storage medium, such as a hard disk, magnetic data storage, optical data storage, memory, etc., and may be accessed by a playback application (referred to as a player) to output (or “playback”) the recorded time-varying content (e.g., output the video, audio, etc.). As an example, the content of recorded time-varying content files may be presented to users via a media player application executing on a processor-based device, such as a personal computer (PC), laptop computer, mobile telephone, portable media player devices (e.g., iPod, etc.). Various media player applications are well known for reading recorded time-varying content files (e.g., multimedia files) and presenting their content to a user, including as examples ADOBE SYSTEMS INCORPORATED'S FLASH PLAYER™ (e.g., ADOBE® FLASH PLAYER 9), MICROSOFT WINDOWS MEDIA PLAYER™ developed by Microsoft Corporation, QUICKTIME™ developed by Apple computer, and REAL PLAYER™ developed by Real Networks. Also, various file formats have been developed for recorded time-varying content (such as multimedia file formats), including as examples such well-known video compression formats as MPEG (Moving Picture Experts Group, including MPEG-1, MPEG-2, and/or MPEG-4, as examples), H.261. H.262, H.263, H.264, AVS, Bink, Dirac, Indeo, MJPEG, Real Network's RealVideo format, Theora, VC-1, VP6, VP7, WMV (Windows Media Video), and QUICKTIME'S MOV format; and including as examples such well-known audio compression formats as MPEG-1 Layer 3 (MP3), MPEG-1 Layer 2, AAC, HE-AAC, G.711, G.722, G.722.1, G.722.2, G.723, G.723.1, G.726, G.728, G.729, G.729.1, G.729a, AC3, Apple Lossless, ATRAC, FLAC, iLBC, Monkey's Audio, μ-law, Musepack, Nellymoser, Real Network's RealAudio format, SHN, Speex, Vorbis, Way Pack, WMA, and TAK. Further, in some instances, a recorded time-varying content file may be implemented using a media container format such as one of the well-known media container formats of 3GP, ASF, AVI (Audio Video Interleave), DMF, DPX, FLV, Matroska, MP4, MXF, NUT, Ogg, Ogg Media, QuickTime, RealMedia, VOB, AIFF, AU, and WAV.
Various authoring tools are known for use in aiding an author in creating, editing, and/or otherwise authoring recorded time-varying content. Such authoring tools typically comprise a computer-executable software program that comprises computer-executable instructions (or code) stored to a computer-readable medium, which when executed by a processor-based device (“computer”) causes the computer to perform operations described herein for such authoring tools. For instance, the authoring tools often present a user interface to a display with which a user (e.g., “an author”) can interact to author (e.g., create, edit, etc.) a recorded time-varying content file. Examples of such authoring tools known in the art include ADOBE FLASH™, ADOBE FLASH MEDIA ENCODER, ADOBE AFTER EFFECTS™, ADOBE AUDITION™, ADOBE PHOTOSHOP™, ADOBE PREMIERE™, ADOBE VISUAL COMMUNICATOR™, and MACROMEDIA DIRECTOR® MX 2004.
Certain authoring tools may aid an author in editing the actual content of a recorded time-varying content file, such as by cutting out portions of the content (e.g., removing frames of a video), rearranging frames in the content, generating portions of the content (e.g., recording video and/or audio, performing computer-generated animation, etc), and/or performing other editing of the content. In addition, certain authoring tools may aid an author in editing certain characteristics of the recorded time-varying content file, such as the file format in which the content is stored, the resolution of the content, the bit rate for encoding and/or playback of the content, the codec used for encoding the content to a particular type of recorded time-varying content file, etc. Other types of authoring operations may be supported by various authoring tools, as are well known in the art.
One popular type of authoring operation that many authoring tools support is the insertion of cue points into a recorded time-varying content file. In general, a cue point is a defined point or marker that can be employed to trigger some event, such as an event that may be external to the recorded time-varying content. Thus, cue points provide invisible markers within a recorded time-varying content file which may be used to trigger such events as synchronization (e.g., synchronizing graphics, subtitles, etc., relative to particular points within the recorded time-varying content), providing navigation options (e.g., marking designated “chapters” or other sections within the recorded time-varying content to enable easy jumping to those marked chapters or sections), and/or loading other content (e.g., loading other video files, etc). As one example, an author may insert a cue point at a point within a video file to trigger subtitle text to be presented to a display at that point of playback of the video. As another example, when authoring video for a television program, an author may insert a cue point within such video file to designate when the video is to break to commercial (e.g., to trigger loading of another video file that presents the commercial content). Thus, authoring tools allow an author to insert a cue point at any point along a recorded time-varying content's reference playback timeline and/or at any given frame of the recorded time-varying content.
As an example, video cue points are supported in ADOBE's FLASH Professional 8, where an author can use cue points in conjunction with the FLASH FLVPlayback component to coordinate the playback of a video clip with accompanying interactive content. For instance, an author can use FLASH Professional 8 to create a FLASH presentation that has video playing in one area of the screen, wherein the author can insert cue points within the video to tigger other actions such as presentation of descriptive text and/or graphics in other areas of the screen at designated points during the playback of the video. Thus, such use of cue points enables the author to create richer and/or more interactive content.
Cue points can be inserted directly into a FLASH Video (FLV) file, for example, using either the FLASH Video Encoder or the Video Import Wizard of FLASH Professional 8. An author can also add and remove cue points with ActionScript using the addCuePoint and removeCuePoint methods.
Cue points are implemented as some type of data structure that defines the cue point properties. For instance, generally each cue point has a name and time value that designates the respective point in time within the recorded time-varying, content (e.g., video) file at which the cue point occurs. The cue point reference time may be specified in hour:minute:second:millisecond format. In some instances, the cue point reference time may be designated based on a corresponding frame of the recorded time-varying content, rather than or in addition to the reference time at which such frame occurs within the recorded time-varying content. For instance, an author may, in some instances view a video frame-by-frame, and insert a cue point at the occurrence of a specific frame.
The cue point data structure may further comprise data indicating a corresponding “type” of the cue point. As discussed further herein, different types of cue points may be defined, such as navigation cue points, event cue points and ActionScript cue points.
Further, a cue point data structure may comprise parameter data that specifies an action to be triggered upon the cue point being encountered. As examples, the parameter data may specify the subtitle text that is to be presented upon the corresponding cue point being encountered, or the parameter data may specify the name of another file (e.g., another video or audio file) to be launched upon the corresponding cue point being encountered.
FIGS. 1A-1B show an exemplary system 10 of the prior art in which a user interface of an authoring application 12 is provided to enable a user to insert a cue point into a video. As shown in FIG. 1A, system 10 comprises a processor-based computer 11, such as a personal computer (PC), laptop computer, server computer, workstation computer, etc. In this example, an authoring tool (or “application”) 12 is executing on such a computer 11 with which a user may interact to author recorded time-varying content, such as the exemplary video being output to a preview display 101. Authoring tool 12 comprises computer-executable software code stored to a computer-readable medium that is readable by a processor of computer 11 and, when executed by such processor, causes computer 11 to perform the various operations described further herein for such authoring tool 12. Examples of such, an authoring tool 12 known in the art include the ADOBE FLASH™, ADOBE FLASH MEDIA ENCODER, ADOBE AFTER EFFECTS™, ADOBE AUDITION™, ADOBE PHOTOSHOP™, ADOBE PREMIERE™, ADOBE VISUAL COMMUNICATOR™, and MACROMEDIA DIRECTOR® MX 2004 authoring applications. The exemplary interface shown as being provided by authoring tool 12 in FIGS. 1A-1B corresponds to that provided by the Flash Video Import Wizard in FLASH Professional 8, with which an author may interact to define cue points for a video as described further hereafter.
In me example of FIGS. 1A-1B, an author interacts with authoring tool 12 to perform authoring operations for a video, such as those operations mentioned above. In particular, in this illustrative example the author interacts with authoring tool 12 to insert a cue point in the video. The exemplary interface of authoring tool 12 provides a content preview display 101 which allows the author to view portions of the content. For instance, the author can “scrub” (or move) along the video's reference timeline 102 to view corresponding parts of the video content. The user may view the video frame-by-frame, in slow-motion playback, etc.
Additionally, certain characteristics of the video file may be defined by the author, such as by defining via input box 106 the quality (e.g., video data rate), video codec, audio codec, audio data rate, etc. Thus, these characteristics of the video file may be modified in addition to or instead of modifying the actual content (e.g., frames) of the video.
A cue point insertion interface 107 is also provided. In FLASH Professional 8, such cue point insertion interface 107 may be accessed by clicking the “Show Advanced Settings” from the Encoding panel of the Video Import Wizard. The author can then select the cue points tab to be presented the cue points interface 107, as shown. On the reference timeline 102, a playback, head indicator 105 is shown, which may be used to locate a specific frame in the video in which the author wants to embed a cue point. For greater precision, the author may select the playback head 105 and use the left and right arrow keys on a keyboard to locate specific points within the video's content. Thus, the video preview window 101 enables an author to visually identify points in the video at which to insert a cue point. The author can also use the elapsed time counter (as indicated by elapsed time indicators 103 and 104 that are located beneath the video preview window 101) to locate specific points in time at which to embed cue points.
When the playback head 105 is positioned on the desired frame, the author can activate (e.g., click with a mouse) the Add Cue Point button 108. FLASH then embeds a cue point on that frame of the video. As shown in FIG. 1B, responsive to the user activating the add cue point button 108, a new cue point 110 is identified in the cue point list 120. The new cue point 110 is defined by certain data, such as a name 111 (e.g., “New Cue Point” in this example), time 112 (e.g., 00:00:17.796 in this example), and type 113 (e.g., an event or navigation type of cue point).
Time 112 identifies the time during playback of the video (e.g., the time on the videos reference timeline 102) when the corresponding event will be triggered by the cue point 110. The types 113 of the cue point may be event or navigation, as examples. Event cue points may be used to trigger ActionScript methods when the cue point is reached. Event cue points enable the author to synchronize the video playback to other events within the FLASH presentation. Navigation cue points may be used for navigation and seeking, and can also trigger ActionScript methods when the cue point is reached. Embedding a navigation cue point may insert a keyframe at that point in the video clip.
In this example, a cue point marker 109 is displayed on the slider control (of reference timeline 102) at the point where the cue point 110 was embedded. The author can use the cue point marker 109 to adjust the placement of the cue point. That is, the author may adjust the placement of the cue point 110 within the video by dragging the corresponding cue point marker 109 to the desired point on timeline 102.
Additionally, the cue point data structure typically comprises parameter data that defines an action to be triggered. As examples, the parameter data may specify the subtitle text that is to be presented upon the corresponding cue point being encountered, or the parameter data may specify the name of another file (e.g., another video or audio file) to be launched upon the corresponding cue point being encountered. Thus, for cue point 110, the author may add one or more parameters into parameter interface 114 by activating insert parameter button 115. For any such parameter, the author may define a corresponding name (in name field 116) and value (in value field 117). For instance, a parameter named “End Subtitle” may be defined whose value specifies the text “The End” to be presented at a corresponding point near the end of the video at which cue point marker 109 may be placed. The parameters may be passed to the cue point event handler as members of the single parameter object.
Traditionally, the cue point data structure defining such information as the cue point's name, time, type, and parameters is embedded within an encoding of a recorded time-varying content file (e.g., video file) in a proprietary (e.g., binary) format. Because the cue point data structure is embedded in the encoded time-varying content file, reuse of a cue point data structure when authoring (e.g., editing) other time-varying content files has traditionally not been available. As a result, inefficiency arises in many instances because the author is required to re-define cue points for each recorded time-varying content file that is being authored on a case-by-case basis. Further, because the cue point data structure has traditionally been in a proprietary format, access to such cue point data structure (e.g., for editing it, etc.) has been limited to an authoring tool that is coded to recognize such proprietary format.
FIG. 2 shows a block diagram of an exemplary system 20 in which an authoring tool 12 may be used to embed a cue point at a desired point in a video, such as described above with FIGS. 1A-1B. In this example, suppose an author uses authoring tool 12 to edit a first recorded time-varying content file 22, including inserting cue points at one or more points in the recorded time-varying content file 22. The cue point data structure 201, including the cue point's name, time, type, and parameters 202, is embedded (in a proprietary format of authoring tool 12) within the encoded time-varying content file 22 that is created. Now, further suppose that the author uses authoring tool 12 to edit a second recorded time-varying content file 23, including inserting cue points at one or more points in the recorded, time-varying content file 23. The cue point data structure 203, including the cue point's name, time, type, and parameters 204, is embedded (in a proprietary format of authoring tool 12) within the encoded time-varying content file 23 that is created. The cue point data structures 201 that were previously defined for recorded time-varying content file 22 are not available for reuse in authoring recorded time-varying content file 23. For instance, when working with a given recorded time-varying content file, authoring tool 12 does not allow an author to recall previously-defined cue point data structures that may have been defined previously for a different recorded time-varying content file. Such lack of reusability leads to inefficiencies by author, as discussed further below.
Suppose, as an example, that an author uses authoring tool 12 to edit a first video 22, and the author interacts with authoring tool 12 in the manner discussed above with FIGS. 1A-1B to specify cue points at various points within the first video 22. Those cue points are defined by corresponding cue point data structures 201 that are embedded in the first video file 22. So, the user creates/edits video 22, inserts the cue points, and then encodes the video to a desired encoded file type, resulting, in encoded video 22. Once encoded, the cue point data structures 201 are not reusable in authoring other videos.
For example, suppose the author then desires a new version of the video. For instance, the first video file 22 may be too long, or its quality may be too poor, or it may be too big in size and/or uses up too much bandwidth to play back, etc. The author may create a new version of the video, such as video 23, to correct these problems. While the cue points 201 previously defined for the first video 22 may be again desired for the new video 23, such cue points are not reusable structures, and thus the author is required to again perform the actions discussed above with FIGS. 1A-1B to re-define the desired cue points for the new video 23, resulting in cue point data structures 203 that are embedded in such video 23.
As another example, in many cases recorded time-varying content is desired to follow a definite structure or format. For instance, television shows follow a designated structure in which they have an opening section, which is everything that plays before the opening credits, and there is typically a break to commercial right after the opening credits, then there will be a first segment of the show followed by another commercial break at some point (e.g., at approximately 15 minutes) into the show, followed by another segment of the show, etc. Thus, the cue points inserted for such a television show follow a similar pattern for each episode of the show, but may need to be adjusted slightly from one episode to the next (e.g., the exact cue point for breaking for commercial at the end of a first segment of the show may not be at an identical time point for every episode of a show). However, traditionally an author using authoring tool 12 is required to repeat the actions for defining/inserting the cue points for each episode of a show (e.g., for a first episode of a show 22, and a second episode of the show 23), as the cue points defined for a first episode are not reusable in authoring a second episode 23.
Accordingly, a desire exists for improved technique for managing cue point data structures.